Skip to content

Conversation

@Fottas
Copy link
Contributor

@Fottas Fottas commented Nov 26, 2025

Summary
Add HTTP client configuration support (timeouts, SSL, etc.) for Spring AI model providers where applicable. Each provider's configuration depends on its underlying HTTP client implementation.

Changes
This PR adds HTTP client configuration properties for the following providers:

DeepSeek

spring:
  ai:
    deepseek:
      connect-timeout: 10s
      read-timeout: 60s

Anthropic

spring:
  ai:
    anthropic:
      connect-timeout: 10s
      read-timeout: 90s

(Add other models you modified...)

Notes

  • Configuration properties vary by provider based on their HTTP client implementation
  • All properties are optional and backward compatible
  • Providers using RestClient support standard timeout and SSL configuration
  • Other providers may have different available options depending on their HTTP client

Resolves

#4934

@Fottas Fottas force-pushed the model-http-timeout branch from 0cfb84c to 5cafbb2 Compare November 26, 2025 11:25
@Fottas Fottas force-pushed the model-http-timeout branch from 785b410 to 109c099 Compare November 27, 2025 00:57
@Fottas Fottas force-pushed the model-http-timeout branch from fe18cd3 to 1f0cdf8 Compare November 27, 2025 11:27
@Fottas
Copy link
Contributor Author

Fottas commented Nov 27, 2025

Summary

This PR adds comprehensive HTTP client configuration support (timeouts, SSL, redirects) for 12 Spring AI model providers. The implementation is based on each provider's underlying HTTP client, ensuring consistency with Spring Boot's configuration patterns while maintaining backward compatibility.

Scope of Changes

Modified Providers (12 total)

  1. DeepSeek - Chat model
  2. Anthropic - Chat model
  3. Azure OpenAI - Chat, Embedding, Audio, Image (multi-modal)
  4. ElevenLabs - Audio model
  5. Google GenAI - Chat and Embedding models
  6. MiniMax - Chat and Embedding models
  7. Mistral AI - Chat, Embedding, Moderation, OCR models
  8. OCI GenAI - Oracle Cloud Infrastructure GenAI
  9. Ollama - Local model support
  10. OpenAI - Complete suite (Chat, Embedding, Audio, Image, Moderation)
  11. Stability AI - Image generation
  12. ZhiPu AI - Chat, Embedding, Image models

Providers Not Modified

  • Vertex AI - Uses Google Cloud SDK with SDK-managed HTTP client
  • PostgresML Embedding - Database connection, no HTTP client
  • Transformers - Local model inference, no HTTP calls
  • Hugging Face - Being migrated to RestClient by another contributor (avoiding duplicate work)

Implementation Approach

Pattern 1: RestClient-based Providers (Recommended)

Applied to 10 providers using Spring RestClient with RestClientCustomizer for configuration injection.

Providers: DeepSeek, Anthropic, ElevenLabs, Google GenAI, MiniMax, Mistral AI, Ollama, OpenAI, Stability AI, ZhiPu AI

Key Components:

  1. ConnectionProperties Extension

    • Added nested HttpClientConfig inner class
    • Supports connectTimeout, readTimeout, redirects, sslBundle
    • Provides toHttpClientSettings() method for Spring Boot standard conversion
  2. RestClientCustomizer Implementation

    • Created dedicated XxxRestClientCustomizer classes
    • Uses ClientHttpRequestFactoryBuilder to build request factory
    • Non-invasive configuration application preserving user customizations
  3. AutoConfiguration Integration

    • Registers RestClientCustomizer bean
    • Supports spring.ai.xxx.http-client.enabled toggle
    • Conditional configuration, enabled by default

Configuration Example:

spring:
  ai:
    deepseek:
      api-key: ${DEEPSEEK_API_KEY}
      http-client:
        enabled: true
        connect-timeout: 10s
        read-timeout: 60s
        redirects: follow
        ssl-bundle: my-ssl-bundle

Pattern 2: Azure SDK-based Providers

Applied to 2 providers using Azure/OCI SDKs with direct timeout configuration in ConnectionProperties.

Providers: Azure OpenAI, OCI GenAI

Key Components:

  1. ConnectionProperties Extension

    • Added connectTimeout and readTimeout properties directly
    • Applied during API client construction
  2. Client Builder Integration

    • Sets timeout parameters when creating Azure/OCI clients
    • Uses SDK-native timeout configuration methods

Configuration Example:

spring:
  ai:
    azure:
      openai:
        api-key: ${AZURE_OPENAI_API_KEY}
        endpoint: ${AZURE_OPENAI_ENDPOINT}
        connect-timeout: 10s
        read-timeout: 90s

Key Design Decisions

  1. Backward Compatibility

    • All configuration properties are optional
    • Can be completely disabled via enabled property
  2. Unified Configuration Structure

    • Follows Spring Boot's HttpClientSettings standard
    • Property naming consistent with Spring Boot conventions
    • SSL Bundle integration support
  3. Non-invasive Implementation

    • Uses RestClientCustomizer instead of direct RestClient modification
    • Preserves any existing user customizations
    • Supports user override via conditional annotations
  4. Comprehensive Test Coverage

    • Added configuration property tests for each provider
    • Validates default and custom values
    • Integration tests verify actual HTTP behavior

Code Statistics

  • Total Files Modified: 59
  • Lines Added: 1,785
  • Lines Removed: 87
  • New Tests: Configuration property tests and integration tests for all providers

Main File Types Changed

  1. AutoConfiguration classes - Added RestClientCustomizer bean registration
  2. ConnectionProperties classes - Added HttpClientConfig configuration class
  3. RestClientCustomizer classes - New dedicated customizer implementations
  4. Test classes - Added configuration property validation and integration tests

Commit History

Chronological commit log:

  1. 265ab8a9 - feat: DeepSeek: support HTTP client timeout configuration
  2. 7b889293 - perf(deepseek): Adjust the configuration timeout method
  3. 29eb9516 - perf(anthropic): support HTTP client timeout configuration
  4. 5cafbb22 - perf(azure-openai): support HTTP client timeout configuration
  5. 2b0dc840 - perf(elevenLabs): support HTTP client timeout configuration
  6. f2930947 - perf(google-genai): support HTTP client timeout configuration
  7. 109c0994 - fix(google-genai): Fix specification errors
  8. 182a2c43 - perf(miniMax): support HTTP client timeout configuration
  9. 6f788b39 - fix(minmax): Configuration error adjustment
  10. 8c7cb3c8 - perf(mistralAi): support HTTP client timeout configuration
  11. ebd61d81 - perf(oci-genai): support HTTP client timeout configuration
  12. 719c239c - perf(ollama): support HTTP client timeout configuration
  13. 4b1b566b - perf(openai): Support HTTP client timeout configuration and nested way using HttpClientSettingsProperties
  14. 1f0cdf8e - perf(stability-ai): Support HTTP client timeout configuration
  15. 6a780cc9 - perf(zhipu-ai): Support HTTP client timeout configuration

Usage Examples

Basic Configuration

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      http-client:
        connect-timeout: 15s
        read-timeout: 120s

Advanced Configuration (SSL + Redirects)

spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      http-client:
        connect-timeout: 10s
        read-timeout: 90s
        redirects: follow
        ssl-bundle: custom-ssl

Disable Custom Configuration

spring:
  ai:
    deepseek:
      api-key: ${DEEPSEEK_API_KEY}
      http-client:
        enabled: false  # Use default RestClient configuration

Testing

All modified providers include:

  1. Default Configuration Tests - Verify default timeout values
  2. Custom Configuration Tests - Verify configuration properties are correctly applied
  3. Disabled Configuration Tests - Verify behavior when enabled=false
  4. Integration Tests - Verify timeout behavior in actual HTTP calls

Related Issue

Resolves #4934

Future Work

  1. Apply same HTTP configuration pattern to Hugging Face provider once RestClient migration is complete
  2. Consider adding similar support for future providers
  3. Documentation updates: Add HTTP configuration examples for each provider

Notes

  • This PR does not change any existing API behavior
  • All configurations are optional with sensible defaults
  • No impact on existing users who don't use these configurations
  • Configuration properties follow Spring Boot naming conventions for ease of use

Document HTTP client configuration options for 12 model providers,
enabling users to customize timeouts, connection pools, and SSL settings.

Changes:
- Add HTTP client properties documentation for RestClient-based providers
  (DeepSeek, Anthropic, OpenAI, Ollama, MiniMax, Mistral AI, ZhiPu AI,
  Stability AI, ElevenLabs)
- Add timeout and connection pool properties for Azure OpenAI
- Add timeout and connection properties for Google GenAI
- Add timeout and async thread properties for OCI GenAI

All properties are optional with no default values, allowing users to
configure HTTP client behavior based on their specific requirements.

Signed-off-by: yinh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant